Search for: All records

Automated interpretation of ultrasound imaging of the heart (echocardiograms) could improve the detection and treatment of aortic stenosis (AS), a deadly heart disease. However, existing deep learning pipelines for assessing AS from echocardiograms have two key limitations. First, most methods rely on limited 2D cineloops, thereby ignoring widely available Spectral Doppler imaging that contains important complementary information about pressure gradients and blood flow abnormalities associated with AS. Second, obtaining labeled data is difficult. There are often far more unlabeled echocardiogram recordings available, but these remain underutilized by existing methods. To overcome these limitations, we introduce Semi-supervised Multimodal Multiple-Instance Learning (SMMIL), a new deep learning framework for automatic interpretation for structural heart diseases like AS. During training, SMMIL can combine a smaller labeled set and an abundant unlabeled set of both 2D and Doppler modalities to improve its classifier. When deployed, SMMIL can combine information from all available images to produce an accurate study-level diagnosis of this life-threatening condition. Experiments demonstrate that SMMIL outperforms recent alternatives, including two medical foundation models. 

Automated interpretation of ultrasound imaging of the heart (echocardiograms) could improve the detection and treatment of aortic stenosis (AS), a deadly heart disease. However, existing deep learning pipelines for assessing AS from echocardiograms have two key limitations. First, most methods rely on limited 2D cineloops, thereby ignoring widely available Spectral Doppler imaging that contains important complementary information about pressure gradients and blood flow abnormalities associated with AS. Second, obtaining labeled data is difficult. There are often far more unlabeled echocardiogram recordings available, but these remain underutilized by existing methods. To overcome these limitations, we introduce Semi-supervised Multimodal Multiple-Instance Learning (SMMIL), a new deep learning framework for automatic interpretation for structural heart diseases like AS. During training, SMMIL can combine a smaller labeled set and an abundant unlabeled set of both 2D and Doppler modalities to improve its classifier. When deployed, SMMIL can combine information from all available images to produce an accurate study-level diagnosis of this life-threatening condition. Experiments demonstrate that SMMIL outperforms recent alternatives, including two medical foundation models. 

Abstract The human brain development experiences a complex evolving cortical folding from a smooth surface to a convoluted ensemble of folds. Computational modeling of brain development has played an essential role in better understanding the process of cortical folding, but still leaves many questions to be answered. A major challenge faced by computational models is how to create massive brain developmental simulations with affordable computational sources to complement neuroimaging data and provide reliable predictions for brain folding. In this study, we leveraged the power of machine learning in data augmentation and prediction to develop a machine-learning-based finite element surrogate model to expedite brain computational simulations, predict brain folding morphology, and explore the underlying folding mechanism. To do so, massive finite element method (FEM) mechanical models were run to simulate brain development using the predefined brain patch growth models with adjustable surface curvature. Then, a GAN-based machine learning model was trained and validated with these produced computational data to predict brain folding morphology given a predefined initial configuration. The results indicate that the machine learning models can predict the complex morphology of folding patterns, including 3-hinge gyral folds. The close agreement between the folding patterns observed in FEM results and those predicted by machine learning models validate the feasibility of the proposed approach, offering a promising avenue to predict the brain development with given fetal brain configurations.